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ATTORNEY'S DOCKET NO. MS1-288US 



TECHNTCAL FIELD 

This invention relates to a system and method for synchronizing streaming 
content with enhancing content. More particularly, this invention relates to a 
system that announces transmission of upcoming enhancing content that is to be 
associated with the streaming content, such as video or a television signal. The 
system further encompasses timely reception of the enhancing content and 
displaying it in conjunction with the streaming content through use of triggers. 

BACKGROUND 

Video content programs, such as TV shows and pay-per-view movies, are 
delivered to a viewer as a continuous data stream. Programs are most commonly 
distributed using a wireless broadcast system or a cable system. In the first 
instance, the programs are broadcast over a wireless network and received at 
individual homes through an antenna or satellite dish. In the latter case, the 
programs are transmitted over cable to set-top boxes resident in the viewers' 
homes. 

In traditional broadcast distribution systems, there is no opportunity for 
interactive control of the content by the viewer. The viewer simply has an option 
to watch the program, change to a different program, or turn off the television. 
However, as consumers have learned from playing video games on their 
televisions, non-interactive viewing is not nearly as fun or sensory rich as 
interactive entertainment. 

To enhance the traditional way of viewing television, there has been some 
effort toward the production of interactive programming content. As presently 
contemplated, additional interactive content is created to enhance the existing 
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traditional program. This supplemental content is played along with the 
continuous video stream to enable viewers to interact with the program in a more 
involved manner than simply watching it. The supplemental content might, for 
example, ask the viewer questions about the episode, or play games with the 
viewer that relate to the show, or describe behind-the-scenes aspects of making the 
program, or provide links to stores that sell merchandise sponsored by the show. 
In addition, the content may not be tied to a particular program, but instead be used 
to convey general information, such as tickers for news headlines, weather 
information, sports scores, and so forth. 

The Internet is rapidly emerging as a means for supplying interactive 
content. Resources available on the Internet are most commonly presented as 
HTML (hypertext markup language) pages. Users can browser through pages of 
information via the World Wide Web ("Web") and receive multimedia- like 
experiences with video, audio, images, text, and other multimedia rich resources. 
Newer technologies are combining the television and Internet experiences to 
provide interactive content. Set-top boxes, for example, are being configured to 
support Web browsers that enable access to Web content in addition to receiving 
traditional television signals. As an example, WebTV Networks, Inc. 
manufactures and sells a browser-based set-top box (STB) having a television 
tuner, a browser, and a standard network interface device that enables 
simultaneously viewing of both television programs and web programming. 

Newer generations of personal computers (PCs) are being implemented 
with broadcast reception capabilities, allowing them to receive and display 
television signals. Such PCs are equipped with a television tuner and software for 
television reception. A Windows-brand operating system from Microsoft 
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Corporation, for example, includes software that supports a number of interactive 
services and reception of broadcast data and content feeds. 

These newer interactive television technologies combine the power of 
traditional television with the compelling interactivity of the Internet. One 
common approach today is to provide the enhancing content via the vertical 
blanking interval (VBI) of the streaming television content. The VBI is a non- 
viewable portion of the television signal. In the future, enhancing content may 
also be provided over satellites (e.g., DVB, DSS, etc.) and as part of digital 
television (DTV) feeds. 

Unfortunately, there remains a significant hurdle concerning creation and 
distribution of interactive programs. TV programs are typically broadcast as a 
continuous data stream. Attempting to synchronize the enhancing content, 
regardless of how that content is delivered (i.e., via the VBI, satellite feeds, etc.), 
with the streaming content poses a difficult design challenge. 

The inventors have developed a system and method that address this 
challenge. 

SUMMARY 

This invention concerns a client-server architecture that synchronizes 
streaming content with enhancing content via pre-announced triggers. 

According to one implementation, the architecture includes server-side 
components that form announcements containing information specifying how and 
when to receive upcoming triggers. The announcements contain such information 
as an identification of the sender, a broadcast locator (e.g., URL, channel, 
frequency, etc.) at which the triggers and enhancing data files are to be broadcast 
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or multicast, a time when the triggers and data files are to be sent, and a protocol 
used to transmit the triggers and data files. The server transmits the 
announcements to a general broadcast or multicast IP (Internet Protocol) address. 

Client-side components include a listener to monitor the IP address for the 
announcements and a filter to extract selected ones of the announcements. 
Selected announcements are stored in an electronic programming guide (EPG) 
database in correlation with the streaming content programs to indicate that the 
associated programs are interactive. 

When a user tunes to an interactive program, the client opens a container 
HTML page that contains controls to receive the streaming content program and to 
extract the announcements from the guide database. One of the controls monitors 
an IP address at particular times specified by a selected announcement to receive 
the triggers corresponding to the interactive program. 

The server delivers the triggers at times synchronized to the streaming 
content. When triggers arrive, a client control processes the triggers to coordinate 
presentation of the enhancing content with the streaming content. This processing 
may involve invoking one or more scripts in the container page or performing 
general navigation in the container page. 

The client-server architecture also enables delivery of headlines and other 
data items employed in a ticker using pre-announced and timely broadcast triggers. 
The data items are carried by the triggers and accumulated into an array. The 
container page inserts the array into a displayed ticker. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Fig. 1 is a diagrammatic illustration of a client-server architecture for 
synchronizing streaming content with enhancing content using pre-announced 
triggers. 

Fig. 2 is a block diagram of an exemplary client computing unit. 

Fig. 3 shows an exemplary configuration of a client software architecture 
implemented on the client computing unit. 

Fig. 4 shows a flow diagram showing steps in a method for synchronizing 
streaming content with enhancing content through use of announcements and 
triggers. 

Fig. 5 shows an exemplary HTML page that combines streaming content 
with enhancing content. It also illustrates use of triggers to carry content for 
tickers. 

DETAILED DESCRIPTION 

This invention concerns an architecture for synchronizing streaming content 
with enhancing content. The architecture governs announcement of upcoming 
enhancing content, reception of the enhancing content when it arrives, and 
association of the enhancing content with the streaming content using triggers 

The enhancing content consists of a set of data files and triggers that are 
multicast to a pre-defined network address to cause specific actions or events to 
occur on any listening client. The triggers are tied to the associated streaming 
content. The enhancing content can take many forms, including text, images, 
animated images, audio, hyperlinks, controls, executable code, and so forth. 
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Announcements are sent ahead of the triggers to announce when the triggers will 
be broadcast and on what multicast address. 

Another aspect of this invention concerns a way to receive headlines and 
other data items as triggers and accumulate them into a ticker. 

General Architecture 

Fig. 1 shows an architecture 20 that enables delivery of streaming content 
and enhancing content from one or more content providers, as represented by a 
primary content provider 22(1) and a secondary content provider 22(2), to multiple 
clients 24(1), 24(2), 24(3), and 24(4). The Fig. 1 illustration shows many different 
ways to deliver streaming and enhancing content. One way is for the same 
provider to deliver both forms of content directly to the clients over the same 
distribution network. This situation is represented by content provider 22(1) 
serving streaming and enhancing content over network 26 to clients 24(1) and 
24(2). 

Another way is for different providers to deliver the contents separately 
over the same or different networks. This situation is represented by the primary 
content provider 22(1) serving streaming content over network 26 to clients 24(1) 
and 24(2) and the secondary content provider 22(2) serving enhancing content 
over network 28 to the clients 24(1) and 24(2). 

Still another way is for a content provider to deliver its content to an 
intermediary broadcast center, which broadcasts the content to clients. This third 
approach is represented by either content provider 22(1) or 22(2) delivering 
packets to a broadcast center 30, which converts network data packets to broadcast 
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packets and broadcasts the packets over a broadcast medium 32 to clients 24(3) 
and 24(4). 

The networks 26, 28, and 32 represent many diverse types of networks 
ranging from data networks (e.g., Internet, LAN (local area network), WAN (wide 
area network), etc.) to television networks (e.g., cable, satellite, RF, microwave, 
etc.). Technologies used to implement the networks are well known and include 
such technologies as wire-based technologies (e.g., fiber optic, cable, wire, etc.), 
wireless technologies (e.g., satellite, RF, etc.), and switching technologies (e.g., 
ATM (Asynchronous Transfer Mode), Ethernet, etc.). These technologies employ 
different data communication protocols (e.g., TCP/IP, IPX/SPX, DSS, NABTS, 
etc.). One specific implementation involves IP over any one of these protocols 
(e.g., IP/NABTS, IP/DSS, IP/DVB, etc.). For an example of IP/NABTS, please 
refer to co-pending US Patent Application Serial Number 08/726,529, entitled 
"Method for Sending Computer Network Data As Part Of Vertical Blanking 
Interval", which was filed October 7, 1996. 

The primary content provider 22(1) has a streaming content server 34 to 
serve streaming content (e.g., video and audio) to the clients. The content may be 
in many different forms, such as TV programs, movies, animation, music, news, 
documentaries, educational productions, and the like. As an exemplary 
implementation, the streaming content server 34 may be implemented as personal 
computers or workstations running a multitasking, disk-based operating system, 
such as Windows NT from Microsoft Corporation, that are configured as 
continuous media file servers that serve data files at a constant data rate. As an 
alternative to a content server, the primary content provider may be a broadcast 
headend that broadcasts the primary content (e.g., broadcast or satellite TV). 
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The primary content provider 22(1) has an enhancing content server 36 to 
serve supplemental or enhancing content to the clients in order to enhance the 
streaming content served by the streaming content server 34. The enhancing 
content could alternatively be stored with the primary content and served before, 
along with, or after the primary content. For instance, a single server may serve 
both the primary and enhancing content and record the content onto a storage 
medium for later transmission. 

The enhancing content may be text, graphics, video, pictures, sound, or 
other multimedia types, as well as applications or other executable code. 
Examples of enhancing content include trivia questions or games related to the 
program, advertisements, merchandise or other memorabilia, hyperlinks to similar 
programs of similar type or starring the same actor/actress, and so on. 

In the implementation described herein, the enhancing content is 
constructed as a hypertext file, or more particularly as an HTML document (or 
"Web page") which can be rendered by a browser. The HTML document may 
include links to other target resources that supply even more content. In concept, 
the target resource can be virtually any type of object — including executable 
programs, text or multimedia documents, sound clips, audio segments, still images, 
computers, directories, and other hyperlinks. In most Web pages, hyperlink targets 
are files that reside on computers connected to the Internet. However, a hyperlink 
target can also be a particular location within a document, including the document 
that is currently being rendered, as well as to other files that may be locally stored 
at the client. 

The primary content provider 22(1) also implements an announcement 
generator 38 that generates and transmits announcements to notify clients of 
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upcoming transmission of enhancing content. Through the announcements, the 
providers tell the clients what data will be served over the network at a given time 
and how to find that data. 

The secondary content provider 22(2) is similar to the primary content 
provider 22(1) in that it implements an enhancing content server 40 and an 
announcement generator 42. These components function essentially as described 
above with respect to the primary content provider 22(1). 

Enhancing Content Stream 

The content providers serve both streaming content and enhancing content 
to the clients. The architecture employs three elements to implement reception and 
display of enhancing content: announcements, triggers, and data files. The first 
element is an "announcement" that describes the IP address and port on which 
corresponding triggers will be sent for a particular enhancement. The second 
element is a "trigger", which tells the client to perform particular actions (e.g., 
execute a script). The third element is the "data files", which are either packages 
containing interactive content files or the interactive content files themselves. 
Together, these elements form the enhancement stream that is delivered to the 
client. 

Announcements, triggers, and data files are all sent on separate IP 
addresses/ports. Sending an enhancement stream involves transmitting the 
following elements: 

1 . An announcement indicating a trigger address and port. 

2. A special trigger indicating an address and port for file transmission. 
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3. A data file containing interactive content files or individual files. 

4. If necessary, a trigger causing the interactive content to be extracted 
out of a container file. 

5. A trigger to cause the interactive content to be displayed. 

"Announcements" inform the client that the streaming content, such as a 
television show, is interactive. The announcements provide details about the 
enhancements, the show that the content enhances, and the page that contains the 
enhancement's starting point. More particularly, the announcements include 
transmission-related information, such as an identification of the sender, a locator 
(e.g., URL, channel, frequency, etc.) at which the triggers and data files are to be 
transmitted, a time when the triggers and data files are to be sent, and a protocol 
used to transmit the triggers and data files. The announcements might further 
include information pertaining to the content of the transmission, such as a title, a 
type of content (e.g., sports, science fiction, mystery, action, documentary, audio, 
graphical, etc.), a subject matter description, a length of transmission, a rating, 
actor/actress names, and so forth. 

Announcements are based on the Internet standard Session Description 
Protocol (SDP). An announcement is made up simply of lines of text, with 
identifiers on the left of an equal sign and associated text on the right. An 
announcement can be stored as a text file until it is ready to be broadcast. 

There are several types of enhancement announcements. The most 
common types are current announcements, which refer to a show that is currently 
on, and channel announcements, which refer to a given channel. Table 1 illustrates 
a sample announcement. 
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Table 1 



Example Description 

v=0 SDP Version number, must be 0. 

o-enhID 2 1 IN IP4 "enhID" specifies a user identifier. "2" is 
155.55.55.55 the session identifier. "1" is the version 

identifier. "155.55.55.55" is replaced 
with the IP address of the server 
transmitting or encoding the content. As a 
whole this field serves as an identifier for 
an enhancement. 

s=Title Contains the title for the enhancement. 



c=IN IP4 233.17.43.44/3 Indicates the IP address on which triggers 

will be sent. 

t=2208988801 Start and finish time for an enhancement. 

2208990601 



a=EnhID:{9E2E8B20- 

083E-lldl-898F- 

00C04FBBDEBC} 



Identifies the announcement as an 
enhancement announcement (as opposed 
to some other type of announcement). 



m=data 17832 udpO 



Specifies the port used for sending 
triggers. 



"Triggers" are notifications that are sent to clients at specific times during 
an interactive data stream. Triggers cause an action to occur on a client, such as 
causing a script to be executed. In this manner, the enhancing content invoked by 
the triggers is synchronized with the streaming content. 

Each type of trigger has a different format. However, all triggers are 
formatted as strings containing three parts: a key or numerical identifier, the 
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trigger data, and a checksum value. The key is separated from the data by white 
space, typically a space or tab character. The format of a trigger is as follows: 

Key TriggerData[checksum] 

where "Key" is the numerical identifier, "TriggerData" is the trigger data, and 
"checksum" is the checksum value. For example, the following string might be 
transmitted to the client for a navigation or "Nav" trigger: 

M 2 bpc : / /MSNBC/default . htm [BA7E] " 

The preceding example illustrates the syntax of the Nav trigger as it is 
transmitted to a client. This is not the format typically used in writing stream 
language files. The syntax transmitted to clients contains additional information 
that you do not need to specify, namely a checksum value and a fully specified 
Broadcast Architecture URL. The Broadcast Architecture supports several basic 
types of triggers as shown in Table 2. 



Lee & Hayes, PLLC 



12 



MS# 1 1 7032 032499 J 048 MS1-288US PA T APP DOC 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



Table 2 

Key Name Description 

0 Error No action is taken. 

1 Data Receives data using the specified transfer 

protocol. 

2 Nav Displays a page. If a target is specified, for 

example by using a FRAME or DIV tag set, the 
page is displayed in that target. 

3 Nav Displays a page. This key value has the same 

functionality as the previous trigger and has been 
kept for compatibility with earlier versions. 

4 Script Runs the specified script. 
6 - 999 Reserved Reserved for future use. 

1,000 and up UserTrigger Sends a user-defined trigger event. 

Table 3 lists exemplary triggers for the navigation-type, script-type, and 
data-type triggers specified by the Broadcast Architecture. 
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Table 3 



Trigger Type Description Format 

Navigation 

Top Page "Home page" for "2 <bpc://MyEnhancement/toppage.htm>'' 

Navigation enhancements 



Target To navigate to a "2 <itv.htm>[targ:LOGO J^RAME]" 

Navigation named target 



Script 



Execute 
ECMAScript 
(Jscript) in the 
top page 



"4 alert('Hi!');" 

"4 Navigate('VIEW_FRAME\ 

'http://www.microsoft.com');" 



Data 
FTS 



Starts listening " 1 <FTS> 

for FTS data 233.1 7.43 .44:2000&MyEnhancemenf 



CAB 



Unpacks cabinet "1 <CAB> 

(CAB) file bpc://MyEnhancement/toppage.cab" 



Enhancement data files and their dependencies are transmitted to clients 
using a transfer service. One exemplary service is the File Transfer Service (FTS), 
which is a component of NetShow, a Microsoft product used for streaming video 
and data on the Internet and on Intranets using IP Multicast. FTS sends files using 
a transfer mechanism based on IP multicast that includes forward error correction 
(FEC). However, other file transfer services might be employed as well. The 
architecture allows use of different file transfer services as needed. 

Enhancement pages, like Web pages, are typically composed of several 
files. For example, the enhancement file "MyEnh.htm" might contain graphic 
images in Graphics Interchange Format (GIF) and Joint Photographic Experts 
Group format (JPEG), ActiveX controls, FutureSplash animations, and other such 
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files that must be downloaded for the enhancement page to display properly. 
These additional files are referred to as "dependency file" or "dependencies" of the 
enhancement file. 

The enhancement dependency files are stored on the client before the 
enhancement is displayed. Otherwise, the user may end up viewing an incomplete 
enhancement and will perceive the content as broken. There are several ways of 
handling dependencies. One approach is to allocated broadcast bandwidth and 
repeatedly download the dependency files. Another approach is to require the user 
to explicitly download the dependency files, either from a Web site or a portable 
medium. 

Still another approach is to package the enhancement and all its 
dependencies together in a single file. In one implementation, this involves use of 
cabinet (CAB) files. Packing the enhancement files together with its dependencies 
ensures that the user receives a complete all-or-nothing viewing experience 
because they either get it all or nothing, which may be better than getting bits and 
pieces. 

Client System 

Clients 24(l)-24(4) can be implemented in a number of ways, including 
computers that are broadcast enabled, set-top boxes, and computer enhanced 
television units. 

Fig. 2 shows an exemplary configuration of a client 24(1) implemented as a 
broadcast-enabled computer. It includes a central processing unit 50 having a 
processor 52, volatile memory 54 (e.g., RAM), and program memory 56 (e.g., 
ROM, disk drive, floppy disk drive, CD-ROM, etc.). The client 24 (1) has one or 
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more input devices 58 (e.g., keyboard, mouse, etc.), a computer display 60 (e.g., 
VGA, SVGA) to display the video content and enhancing content, and a stereo I/O 
62 for interfacing with a stereo system to play audio content. 

The client 24(1) includes a digital broadcast receiver 64 (e.g., satellite dish 
receiver, RF receiver, microwave receiver, network connection, etc.) and a tuner 
66 which tunes to frequencies of a broadcast network. The tuner 66 is configured 
to receive digital broadcast data in a particularized format, such as MPEG-encoded 
digital video and audio data, as well as digital data in many different forms, 
including software programs and programming information in the form of data 
files. The client 24(1) also has a modem 68 that provides access to the Internet or 
other network. For other implementations, the modem 68 might be replaced by a 
network card, an RF receiver, or other type of port/receiver. 

One example implementation of a broadcast- enabled PC is described in a 
co-pending U.S. Patent Application Serial No. 08/503,055, entitled "Broadcast- 
Enabled Personal Computer," filed January 29, 1996 in the names of Gabe L. 
Newell, Dan Newell, Steven J. Fluegel, David S. Byrne, Whitney McCleary, 
James O. Robarts, Brian K. Moran; William B. McCormick, T.K. Backman, 
Kenneth J. Birdwell, Joseph S. Robinson, Alonzo Gariepy, Marc W. Whitman, 
and Larry Brader. This application is assigned to Microsoft Corporation, and is 
incorporated herein by reference. 

The client 24(1) runs an operating system 70 that supports multiple 
applications. The operating system is preferably a multitasking operating system 
that allows simultaneous execution of multiple applications. One preferred 
operating system is a Windows brand operating system sold by Microsoft 
Corporation, such as Windows 98, Windows CE, or Windows NT or other 
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derivative versions of Windows. It is noted, however, that other operating systems 
may be employed, such as the Macintosh operating system from Apple Computer, 
Inc. and the OS/2 operating system from IBM. 

The client 24(1) further runs a Web browser 72 that is capable of navigating 
hyperlinks and receiving and rendering hypermedia content, such as HTML pages. 
As an alternative to a browser, the client may run a piece of software known as 
"WebTV for Windows", which is available from Microsoft Corporation. 

In addition to the operating system 70 and browser 72, the client 24(1) is 
illustrated with a number of software programs or modules that can be started or 
called to receive and play enhancing content in conjunction with streaming 
content. The modules include an announcement listener 74, one or more filters 76, 
one or more controls 78, and one or more scripts 80. Listener 74 is a specialized 
control that operates on information provided by announcements to receive and 
process triggers. Each program is stored in program memory 56, loaded into 
volatile memory 54 when launched, and executed on the processor 52. These 
software components are described below in more detail with reference to Fig. 3. 

The client 24(1) also has maintains a TV system services (TSS) database 82 
that holds an electronic program guide (EPG) and a spool directory 84 that 
functions as a cache for the TV based services and to hold the enhancing data files. 

Client Software Architecture 

Fig. 3 shows an exemplary configuration of the client software architecture 
90. The streaming and enhancing content may arrive at the client in one of two 
ways. First, a single source (e.g., primary content provider 22(1)) may transmit 
both the streaming content and the enhancing content in a composite data stream. 
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In traditional television broadcast, the enhancing content is inserted into the VBL 
Alternatively, the streaming content may be served from one source (e.g., primary 
content provider 22(1)) while the enhancing content is served from a second 
source (e.g., secondary content provider 22(2)). 

The enhancing content comprises the three elements: announcements, 
triggers, and data files. The three elements are sent at different times to different 
IP addresses and ports. The data files include both the enhancement files and the 
dependency files. 

Receiver 64 receives the composite stream or the dual streams. Receiver 64 
represents one or more receiver types, such as a broadcast receiver, a network 
connection, a cable receiver, and so forth. The receiver extracts the streaming 
content (e.g., video and audio data) and sends it to one or two types of video 
controls 78: a video control, which is used in Web pages that are not hosted in 
WebTV for Windows, and an enhancement video control, which is used in Web 
pages that are hosted in WebTV for Windows. The video controls reside in an 
HTML page 92 that is opened by a browser 72 (or the WebTV for Windows 
software) to support the interactive program. 

As video is received by the video control 78, elements of the enhancement 
stream (announcements, triggers, data files) are received at other components of 
the client. The enhancement announcements are passed through one or more filters 
76, which examine each announcement for a match against a list of programs in 
which the user is interested, or against other types of predefined rales of 
acceptance. The filter(s) 76 retain the announcements of interest, and discard the 
rest. 
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Selected announcements are stored in the TSS database 82, which also 
contains program guide information. The announcements are correlated with the 
video content shows that can be enhanced. In this manner, the presence of 
enhancement data in conjunction with a show listed in the database indicates that 
the show is interactive. 

Enhancement listener 74 handles the announcements stored in the TSS 
database 82 when the user requests the corresponding interactive program. The 
listener 74 uses information from the announcements to prepare to receive triggers. 
The enhancement listener 74 is implemented as an ActiveX control that responds 
to enhancement stream events. Like the video controls, the enhancement listener 
is hosted in an enhancement page 92 displayed by browser 72. 

Depending upon the trigger type, the listener 74 may perform several 
different functions. A data-type trigger is used in facilitating reception of the data 
files. The data-type trigger contains the IP address and port at which the data files 
will be delivered. The enhancing data files are temporarily cached in spool 
directory 84. 

A navigation-type trigger is used for navigation about the container HTML 
page 92. A script-type trigger contains information to invoke one or more scripts 
80 for execution. The scripts use the enhancing data stored in the spook directory 
84 to enhancing the streaming content being played by the video control 78. 

Content Synchronization 

Fig. 4 shows exemplary steps in a method for synchronizing streaming 
content with enhancing content through use of announcements and triggers. At 
step 100, the client receives streaming and enhancing content from one or more 
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content providers. The receiver passes the streaming content to video control 78 in 
an HTML page 72 (step 102). 

Meanwhile, the enhancement listener 74 executes in the background to 
listen for announcements that may accompany streaming content. One or more 
filters 76 register with, and are loaded by, the announcement listener 74 to filter 
out unwanted announcements received in the enhancing content stream (step 104). 
At step 106, announcements surviving the filtering process are stored in TSS 
database 82. The filter 176 calls a loader (e.g., DLL object) to load the 
announcement in the TSS database. The selected announcements are stored in 
correlation with the titles or identities of the streaming content programs to 
indicate that the programs are interactive. 

Each time a user tunes a new channel, browser 72 checks the TSS database 
82 to see if the new show is interactive (step 108). If not (i.e., the "no" branch 
from step 108), the client simply plays the streaming content (step 110). On the 
other hand, if the new show is interactive (i.e., the "yes" branch from step 108) and 
assuming the user has enhancements enabled, the client browser opens a container 
HTML page 92 and displays a special icon indicating that the show is interactive 
(step 112). The container page 92 includes the controls and scripts to render the 
streaming and enhancing content on the display. 

When browser 72 displays an interactive show, the listener 74 listens on the 
IP address and port specified in an earlier announcement stored in the TSS 
database 82 (step 114). The announcement listener 74 monitors the IP stream at 
the address and port for triggers. Triggers are sent at specific times during the 
associated streaming content to cause an action to occur on a client in relation to 
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the streaming content. In this manner, the enhancing content induced by the 
triggers is synchronized with the streaming content. 

When a trigger is received, the listener 74 processes the trigger (step 116). 
Part of the processing determines whether the trigger is a standard trigger, such as 
the Broadcast Architecture defined data-type, navigation-type, and script-type 
triggers, or a user-defined trigger. 

For a data-type trigger, the client is prepared to receive enhancing data files 
according to a particular protocol (step 118). One example of a data-type trigger, 
known as the FTS data trigger, indicates a subdirectory in which to store 
enhancing data files in the spool directory 84 and the IP address and port on which 
files will be transferred. The FTS data trigger uses a "bpc:" URL syntax to 
reference delivered files from HTML. The syntax allows for the use of 
subdirectories. A sample FTS data trigger is given as follows: 

1 <FTS> 233.43. 17.17:17817&MyEnhancement 

The enhancing data files are stored in the MyEnhancement subdirectory of 
the spook directory 84 (step 120). The URL to a referenced HTML page in the 
MyEnhancement subdirectory is given as follows: 

bpc ://MyEnhancement/funstuff .htm 

For a navigation-type trigger, the browser uses the trigger information to 
perform general navigation operations, such as moving to a top of page or 
referencing a target resource (step 122). For a script-type triggers and user-defined 
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triggers, the listener sends an event to the container Web page 92, where they are 
handled by scripts 80 embedded in that Web page (step 124). 

If the triggers involve accessing enhancement data files, the listener 74 uses 
the URL to retrieve the data files from the spool directory 84 in timely manner 
(step 126). The browser then renders the enhancing content in synchronization 
with the streaming content (step 128). 

Trigger-Carried Ticker Information 

Another aspect of this invention concerns a way to receive headlines and 
other information as triggers and accumulate them into a ticker. Triggers holding 
ticker information are accumulated over time and then displayed in a ticker. It also 
provides a way to have text fade-in and fade-out (using Dynamic HTML or 
"DHTML"). 

To demonstrate this technology, Fig. 5 shows an HTML page 150 for the 
MSNBC Web site " www.MSNBC.com " that combines and coordinates both 
streaming content and enhancing content. It includes a video pane 152 within 
which the streaming video data is played. The MSNBC page 150 also has a static 
link 154, such as a link to a home page, weather, or local news. Dynamic links 
156 are edited and managed on a more routine basis, such as daily or hourly, and 
hence change more often than the static link 154. 

The MSNBC page also has an advertising space 158 to display DHTML 
pages used to present advertisements. A DHTML ticker 160 is provided to scroll 
section names, headlines, URLs, schedule information, and so forth. 

The MSNBC enhancement page 150 operates as follows: 
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1 . The enhancement page 1 50 is received and rendered by the browser 72. 

2. The enhancement page 150 receives triggers containing headlines and 
other items for the ticker 160 and stores them as elements in an array. 
The enhancement page 150 also receives triggers for the dynamic links 
156. 

3. The page operates on a timer to cycle through all of the stored triggers, 
fading from one headline to the next. 

4. Each headline has an associated Web link (URL); if the user clicks on a 
headline, a child browser window is created in which the referenced 
page is displayed. 

The ticker data is stored in three arrays defined as follows: 
// MSNBC Headlines 

MSNBC_Category = new MakeArray ( 4 0 ) ; 
MSNBCJJRL = new MakeAr ray ( 4 0 ) ; 
MSNBC_Headline = new MakeArray ( 4 0 ) ; 

The array "MSNBCCategory" stores the headline category; 
"MSNBCJURL" stores the link associated with the headline, and 
"MSNBC_Headline" stores the actual headline text that is displayed in the ticker. 
Each array can store up to forty elements corresponding to ticker headlines. The 
array is initially filled with empty strings. As the ticker cycles through the arrays, 
it ignores any element that is an empty string. To fill out the arrays and thereby 
add headlines to the ticker data, the content provider transmits a trigger that calls 
the following Jscript routine: 
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function ChangeHeadline (tnum, teat , turl , thl) 

{ 

tnum- - ; 

if (tnum >= 0 && tnum < 40) 

{ . 

MSNBC_Category [tnum] = teat; 
MSNBC_URL [tnum] = turl; 
MSNBC_Headline [tnum] = thl; 

} 

} 



The "ChangeHeadline" function inserts the headline "thl" at the array index 
"tnum". The value specified in "tnum" allows the content producer to order the 
headlines in a particular fashion. The ticker cycles through the arrays in numeric 
order, skipping empty array elements. 

Here are some examples of triggers that call "ChangeHeadline". Trigger 
type 4 causes the trigger listener control 74 on the enhancement page to execute 
the script specified in the trigger. 

00:00:05.00 trigger 4 ( 

"top. main. ChangeHeadline (1, 'Brian' , ' http : //msw ' , 1 T 
est 1 Headline ! ); 11 ) only; 

00:00:10.00 trigger 4 ( 

"top. main. ChangeHeadline (10, 'Seattle' , 'http: //msw' 
, 'Test 10 Headline Test 2 Headline Test 2 Headline 
Test 2 Headline Test 2 Headline Test 2 Headline 
Test 2 Headline Test 2 Headline Test 2 
Headline ' ) ; " ) only; 

00:00:15.00 trigger 4 ( 

"top. main. ChangeHeadline (11, 'Seattle', 'http: //msw' 
, 'Test 11 Headline Test 2 Headline Test 2 Headline 
Test 2 Headline Test 2 Headline Test 2 Headline 
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Test 2 Headline Test 2 Headline Test 2 
Headline 1 ) ; " ) only; 

The following trigger will remove the headline from the array at index 1 1 . 
00:00:15.00 trigger 4 ( 

"top.main.ChangeHeadline(ll, 1 ' , 1 ' , ' ' ) ; " ) only; 



The following is an excerpt from an actual stream file as generated by the 
MSNBC tools: 
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bandwi dt h= 3 84 0 0; 

00:00:01.00 trigger 4 ( 
"top . main . ChangeStanding ( 0 , 

' http : //www.msnbc . com/news/itv_redir . asp?http : / /ww 
w.msnbc . com/modules/ starr/def ault . asp ' , 
1 Starr&#39 ; s growing web of investigations 1 ); " ) 
only; 

00:00:02.00 trigger 4 ( 
" top . main . ChangeStanding ( 1 , 

' http : / /www . msnbc . com/news/ itv_redir . asp?ht tp : //ww 
w.msnbc . com/news/180095 . asp ' , 'Sabotage may have 
killed U.S. envoy') ;" ) only; 

00:00:03.00 trigger 4 ( 
"top. main. ChangeStanding (2 , 

1 http : / /www . msnbc . com/news/ itv_redir . asp?http : //ww 
w.msnbc . com/modules/clinics/default . asp ' , 1 Clinic 
violence, state-by-state 1 ); " ) only; 

00:00:04.00 trigger 4 ( 

"top .main. ChangeHeadline (1 , ' 1 , 

' http : //www.msnbc . com/news/itv__redir . asp? ' , 1 1 ) ; " 
) only; 

00:00:05.00 trigger 4 ( 
" top . main . ChangeHeadl ine ( 2 , 
1 Time  &   Again 1 , 

' http : //www.msnbc . com/news/ itv_redir . asp?/news/TIM 
EAGAI1SME ront . asp 1 , ' JFK&#39;s presidential 
nomination. Wednesday, July 15 12:00AM 1 );" ) only; 



Conclusion 

Although the invention has been described in language specific to structural 
features and/or methodological steps, it is to be understood that the invention 
defined in the appended claims is not necessarily limited to the specific features or 
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steps described. Rather, the specific features and steps are disclosed as exemplary 
forms of implementing the claimed invention. 
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CLAIMS 

1. A method for synchronizing streaming content with enhancing 
content comprising the following steps: 

receiving an announcement containing information specifying how and 
when to receive upcoming enhancing content, the announcement being sent at a 
time prior to sending the enhancing content, the enhancing content being 
associated with the streaming content; 

receiving the streaming content; 

receiving the enhancing content according to the information contained in 
the announcement and at a time in synchronization with the streaming content; and 
enhancing the streaming content with the enhancing content. 

2. A method as recited in claim 1, wherein the announcement contains 
parameters selected from a group comprising: a broadcast locator, a time when the 
corresponding enhancing content is to be sent, a protocol, an identity of the 
streaming content that the enhancing content enhances, and a page that contains a 
starting point for the enhancing content. 

3. A method as recited in claim 1, wherein the enhancing content 
comprises triggers and data files, the method further comprising the following 
steps: 

receiving the data files; and 

receiving the triggers at times in synchronization with the streaming 
content, the triggers causing operations involving the data files in order to timely 
introduce the enhancing content with the streaming content. 
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4. A method as recited in claim 3, wherein the enhancing content further 
comprises dependency files that contain instructions to present content contained 
in the data files, the method further comprising the step of delivering the data files 
together with the dependency files in a cabinet (CAB) file format. 

5. A method as recited in claim 1, wherein the receiving steps comprise 
the step of receiving the streaming content and the enhancing content in a 
composite stream of one source. 

6. A method as recited in claim 1, wherein the receiving steps comprise 
the following steps: 

receiving the streaming content from a first source; and 
receiving the enhancing content from a second source different from the 
first source. 

7. A method as recited in claim 1 ? further comprising the step of 
displaying the enhancing content as a ticker. 

8. A computer-readable medium having computer-executable 
instructions for performing the steps of the method as recited in claim 1 . 
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9. A method for synchronizing streaming content with enhancing 
content comprising the following steps: 

forming announcements containing information specifying a first address 
and a time at which to receive upcoming triggers; 

sending the announcements to a second address; 

monitoring the second address to receive the announcements; 

filtering the announcements to retain selected announcements; 

monitoring the first address at the time specified in the selected 
announcements to receive the triggers; and 

processing the triggers to coordinate presentation of the enhancing content 
with the streaming content. 

10. A method as recited in claim 9, wherein the forming step comprises 
the step of creating an announcement to contain parameters selected from a group 
comprising: a broadcast locator, a time when a corresponding trigger is to be 
broadcast, a broadcast protocol, an identity of the streaming content that the 
enhancing content enhances, and a page that contains a starting point for the 
enhancing content. 

11. A method as recited in claim 9, wherein the processing step 
comprises the step of receiving the enhancing content according to a 
predetermined protocol and storing the enhancing content. 
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12. A method as recited in claim 9, wherein the processing step 
comprises the step of navigating within a container HTML page. 

13. A method as recited in claim 9, wherein the processing step 
comprises the step of invoking a script within a container HTML page. 

14. A method as recited in claim 9, further comprising the step of 
sending the enhancing content together with the triggers. 

15. A method as recited in claim 9, further comprising the step of 
delivering the streaming content, the enhancing content, and the triggers from a 
same source. 

16. A method as recited in claim 9, further comprising the following 

steps: 

delivering the enhancing content from a first source; and 
delivering the streaming content and the triggers from a second source 
different from the first source. 

17. A method as recited in claim 9, further comprising the step of 
displaying the enhancing content together with the streaming content. 

18. A method as recited in claim 9, wherein the enhancing content 
comprises a ticker, and further comprising the following steps: 

delivering triggers that contain items for the ticker; 
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displaying the ticker together with the streaming content; and 
presenting the items from the triggers within the ticker. 

19, Computer-readable media distributed at a content provider and a 
client and having computer-executable instructions for performing the steps of the 
method as recited in claim 9. 

20, A method comprising the following steps: 
displaying streaming content; 

monitoring an announcement address to receive announcements pertaining 
to enhancing content for enhancing the streaming content, the announcements 
containing information that specify a trigger address and times at which to receive 
upcoming triggers; 

filtering the announcements to retain selected announcements; 

storing the selected announcements; 

monitoring the trigger address at the times specified in the selected 
announcements to receive corresponding triggers; and 

processing the triggers to cause one of the following events to occur: (1) 
receive one or more data files having the enhancing content, or (2) utilize the 
enhancing content in the data files to enhance the streaming content. 

21, A method as recited in claim 20, further comprising the step of 
caching the data files. 
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22. A method as recited in claim 20, further comprising the step of 
presenting the streaming content and the enhancing content using an HTML page. 

23. A method as recited in claim 20, further comprising the following 

steps: 

using information from the trigger to invoke a script; and 
executing the script to enhance the streaming content. 

24. A method as recited in claim 20, further comprising the following 

steps: 

receiving triggers that contain items for a ticker; 

displaying the ticker together with the streaming content; and 

presenting the items from the triggers within the ticker. 

25. A computer-readable medium having computer-executable 
instructions for performing the steps of the method as recited in claim 20. 

26. A method comprising the following steps: 

monitoring an IP (Internet Protocol) address to receive triggers, each trigger 
containing at least one item for a ticker; 

forming an array of the items received in the triggers; and 
displaying the array within the ticker. 
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27. A method as recited in claim 26, further comprising the step of 
receiving announcements at another IP address, the announcements being used to 
announce upcoming transmissions of the triggers. 

28. A computer-readable system having computer-executable 
instructions for performing the following steps: 

monitoring an announcement address to receive announcements pertaining 
to enhancing content for enhancing the streaming content, the announcements 
containing information that specify a trigger address and a time at which to receive 
upcoming triggers; 

filtering the announcements to retain selected announcements; 

monitoring the trigger IP address at the time specified in one of the selected 
announcements to receive a corresponding trigger; and 

processing the trigger to cause utilization of the enhancing content to 
enhance the streaming content. 

29. A computer-readable system having computer-executable 
instructions for performing the following steps: 

monitoring an IP address to receive triggers, each trigger containing at least 
one item; and 

presenting a ticker that contains the items. 

30. A system for synchronizing streaming content with enhancing 
content, comprising: 

at least one content server; 
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a client; 

the content server being configured to form announcements that contain 
information specifying an IP address and a time at which to receive upcoming 
triggers, the content server sending the announcements to the client; and 

the client being configured to receive the announcements and monitor the 
IP address at the time specified in the announcements to receive the triggers, the 
client processing the triggers to coordinate presentation of the enhancing content 
with the streaming content. 

31 • A system as recited in claim 30, wherein the announcements contain 
parameters selected from a group comprising: a broadcast locator, a time when a 
corresponding trigger is to be broadcast, a broadcast protocol, an identity of the 
streaming content that the enhancing content enhances, and a page that contains a 
starting point for the enhancing content. 

32. A system as recited in claim 30, wherein the client receives the 
enhancing content according to a predetermined protocol and caches the enhancing 
content. 

33. A system as recited in claim 30, wherein the client uses information 
from the trigger to navigate within a container HTML page. 

34. A system as recited in claim 30, wherein the client uses information 
from the trigger to invoke a script within a container HTML page. 
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35. A system as recited in claim 30, wherein the content server delivers 
the streaming content, the enhancing content, and the triggers, 

36. A system as recited in claim 30, wherein the content server delivers 
the enhancing content and the triggers, and further comprising a second content 
server to deliver the streaming content. 

37. A system as recited in claim 30, wherein the triggers contain items 
for a ticker and the client displays the ticker together with the streaming content. 

38. A client software architecture, comprising: 
a video control to play video content; 

a listener to monitor an announcement IP (Internet Protocol) address to 
receive announcements pertaining to enhancing content for enhancing the video 
content, the announcements containing information that specify a trigger IP 
address and times at which to receive upcoming triggers; 

the listener further monitoring the trigger IP address at the times specified 
by the announcements to receive corresponding triggers; and 

a rendering component to present the video content and to enhance the 
video content with the enhancing content according to instructions received in the 
triggers. 
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39. A client software architecture as recited in claim 38, further 
comprising a storage to hold the announcements in correlation with an identity of 
the video content. 

40. A client software architecture as recited in claim 38, further 
comprising a cache to hold the enhancing content. 

41. A client software architecture as recited in claim 38, wherein the 
listener and video control are contained within an HTML page rendered by the 
rendering component. 

42. A client software architecture as recited in claim 41, wherein the 
HTML page also has one or more scripts to process the instructions contained in 
the triggers. 

43. A client software architecture, comprising: 

a first code segment for monitoring a broadcast or multicast IP address to 
receive triggers, each trigger containing at least one item for a ticker; and 

a second code segment for rendering the ticker with the items received via 
the triggers. 
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ABSTRACT 

A client-server architecture synchronizes streaming content with enhancing 
content via pre-announced triggers. The architecture includes server-based 
components to generate announcements containing information specifying how 
and when to receive upcoming triggers that will be transmitted at a later time. The 
server transmits the announcements to a general broadcast or multicast IP address. 
Client-based components monitor the IP address to receive and filter the 
announcements. Selected announcements are stored in a guide database in 
correlation with the streaming content programs to indicate that the programs are 
interactive. When a user tunes to an interactive program, the client opens a 
container HTML page that contains controls to receive the streaming content 
program and to extract the announcements from the guide database. The latter 
control monitors the IP address at the times specified in the selected 
announcements to receive the triggers corresponding to the interactive program. 
The server delivers the triggers at times synchronized to the streaming content. 
When triggers arrive, the client control processes the triggers to coordinate 
presentation of the enhancing content with the streaming content program. The 
triggers may further be used to carry items that fill a ticker being displayed with 
the program. 
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